Structure Discovery in Nonparametric Regression through Compositional Kernel Search
نویسندگان
چکیده
Despite its importance, choosing the structural form of the kernel in nonparametric regression remains a black art. We define a space of kernel structures which are built compositionally by adding and multiplying a small number of base kernels. We present a method for searching over this space of structures which mirrors the scientific discovery process. The learned structures can often decompose functions into interpretable components and enable long-range extrapolation on time-series datasets. Our structure search method outperforms many widely used kernels and kernel combination methods on a variety of prediction tasks.
منابع مشابه
Nonparametric Regression Estimation under Kernel Polynomial Model for Unstructured Data
The nonparametric estimation(NE) of kernel polynomial regression (KPR) model is a powerful tool to visually depict the effect of covariates on response variable, when there exist unstructured and heterogeneous data. In this paper we introduce KPR model that is the mixture of nonparametric regression models with bootstrap algorithm, which is considered in a heterogeneous and unstructured framewo...
متن کاملTHE COMPARISON OF TWO METHOD NONPARAMETRIC APPROACH ON SMALL AREA ESTIMATION (CASE: APPROACH WITH KERNEL METHODS AND LOCAL POLYNOMIAL REGRESSION)
Small Area estimation is a technique used to estimate parameters of subpopulations with small sample sizes. Small area estimation is needed in obtaining information on a small area, such as sub-district or village. Generally, in some cases, small area estimation uses parametric modeling. But in fact, a lot of models have no linear relationship between the small area average and the covariat...
متن کاملRobust nonparametric kernel regression estimator
In robust nonparametric kernel regression context,weprescribemethod to select trimming parameter and bandwidth. Through solving estimating equations, we control outlier effect through combining weighting and trimming. We show asymptotic consistency, establish bias, variance properties and derive asymptotics. © 2016 Elsevier B.V. All rights reserved.
متن کاملHierarchically Compositional Kernels for Scalable Nonparametric Learning
We propose a novel class of kernels to alleviate the high computational cost of large-scale nonparametric learning with kernel methods. The proposed kernel is defined based on a hierarchical partitioning of the underlying data domain, where the Nyström method (a globally low-rank approximation) is married with a locally lossless approximation in a hierarchical fashion. The kernel maintains (str...
متن کاملScalable Structure Discovery in Regression using Gaussian Processes
Automatic Bayesian Covariance Discovery (ABCD) in Lloyd et al. (2014) provides a framework for automating statistical modelling as well as exploratory data analysis for regression problems. However ABCD does not scale due to its O(N) running time for the kernel search. This is undesirable not only because the average size of data sets is growing fast, but also because there is potentially more ...
متن کامل